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THE  PRODUCT  PORM  FOR  THE  INVERSE  IN  THE  SIMPLEX  METHOD 

V 

Oeorge  B«  Dantzlg 
Wm .  Orchard-Htys 

j 

SUMMARY;  When  a  matrix  Is  represented  as  a  product  of  ^elementary” 
matrices,  the  matrix,  its  transpose,  its  inverse  and  inverse  trans— 
pose  are  readily  available  for  vector  multiplication.  By  an 

L  > 

elementary  matrix  is  meant  one  formed  from  the  identity  matrix 
by  replacing  one  column;  thus  an  elementary  matrix  can  be  com¬ 
pactly  recorded  by  the  subscript  of  the  altered  column  and  the 
values  of  the  elements  in  It .  In  the  revised  simplex  method 
both  the  inverse  and  inverse  transpose  of  a  ^basic*^  matrix  are 
needed;  more  significant,  however,  is  the  fact  that  each  itera¬ 
tion  replaces  one  of  the  columns  of  the  basis.  In  the  product 
form  of  representation,  this  change  can  be  conveniently  effected 
by  multiplying  the  previous  matrix  by  an  elementary  matrix;  thus, 
only  one  additional  column  of  information  need  be  recorded  with 
each  iteration.  This  approach  places  relatively  greater  emphasis 
on  reading  operations  than  writing  and  thereby  reduces  com¬ 
putation  time.  Using  the  I.B.M.  Card  Programmed  Calculator, 
a  novel  feature  results;  when  the  inverse  matrix  is  needed  at 
one  stage  and  its  transpose  at  another,  this  is  achieved  simply 
by  turning  over  the  deck  of  cards  representing  the  inverse.  (  ) 
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INTRODUCTION; 

The  slnplex  method  le  an  algorithm  for  determining  values 
for  a  set  of  n  non-negative  variables  which  minimizes  a  linear 
form  subject  to  m  linear  restraints.  D]  may  be 

characterized  briefly  as  a  finite  iterative  procedure.  Bach 
iteration  produces  a  new  special  solution  to  the  restraint  equ^ 
tions  involving  a  subset  of  m  of  the  variables,  only  one  element 
of  the  subset  changing  on  successive  iterations;  the  remaining 

n-ffl  variables  are  equated  to  zero.  The  vectors  of  coefficients 

* 

corresponding  to  the  subset  of  m  variables  are  linearly  indepencH 
ent  and  constitute  a  basis  in  m~dlmension  real  vector  space* 

In  the  original  qimplex  method  [210  (as  coded  for  the  _ 

N.B.S.  S.E.A.C.  Blectronlo  Computer  [4])  or  as  found  in  , 
it  is  required  that  all  the  coefficient  vectors  be  represented 
in  textna  of  the  latest  basis;  since  the  changes  of  basis  are 
step-wise,  a  simple  recursion  relation  suffices  to  alter  the 
representations  on  each  iteration. 

The  revised  simplex  method  differs  from  the  original 

method  in  that  it  uses  the  same  recursion  relations  to  trans— 
foxm  only  the  inverse  of  the  basis  for  each  iteration.  It  has 
been  introduced  to  reduce  the  quantity  of  writing  at  each  step 
(which  it  does  in  general),  and  is  particularly  effective  for 
linear  programming  models  where  the  original  matrix  of  coefficients 
is  largely  composed  of  zeros,  as  for  example,  in  the  transporta¬ 
tion  model  [2c]  or  dynamic  economic  and  production  models  [2b]  • 

If  the  original  method  is  used,  these  zeros  would  be  replaced 
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by  non— zeros  In  tne  successive  iterations  and  this  greatly 
increases  the  computational  effort.  On  the  other  hand,  the 
revised  method  leaves  those  zeros  Intact. 

One  important  feature  of  the  simplex  method  is  concerned 
with  the  criteria  by  which  one  of  the  vectors  in  the  basis  is 
replaced  by  a  vector  not  in  the  basis  to  form  the  basis  of  the 
next  iteration.  When  the  constant  terms  of  the  restraint 
equations  are  not  general,  the  choice  of  the  vector  to  drop 
from  the  basis  may  be  ambiguous  and  an  arbitrary  selection 
(as  pointed  out  In  unpublished  examples  by  Alan  Hoffman  and 
Philip  Wolfe)  may  lead  to  non-convergence.  Several  devices 
exist,  however,  for  perturbing  the  constant  terms  so  as  to 
avoid  this  difficulty.  The  earliest  proposal  along  these  lines 
[2a0  consisted  in  modifying  the  vector  of  constant  terms  by  a 
specially  weighted  combination  of  the  unit  vectcra.  This  approach 
may  be  used  conveniently  both  for  the  revised  and  original  simplex 
[methods,  With  the  original  simplex  method,  there  is  another 

natural  way  to  fonri  the  pertubation  wnich  consists  in  adding  a 
weighted  linear  combination  of  the  column  vectors  to  the  vector 
of  constant  terms.  This  was  suggested  first  by  Orden  and  de¬ 
veloped  Independently  by  Charnes,  [;>]  . 

Although  considerable  attention  has  been  paid  to  the  above 
difficulty  (called  degeneracy ) ,  it  usually  does  not  lead  to  non- 
convergence  .  The  type  of  problems  in  which  it  can  cause  non- 
convergence  appear  to  be  exceedingly  rare.  To  date,  there  have 
been  only  two  examples  and  these  were  artificially  constructed 
for  tnis  purpose.  Accordingly,  the  S.E.A.C.  code  and  the  HAND 
code  use  an  arbitrary  selection  criteria  in  case  of  ambiguity . 
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In  the^e  codes,  a  deliberate  decision  was  made  to  use  a  simple 
code  In  lieu  of  a  more  complex  one  needed  to  cover  a  possible 
case  that  may  never  arise  In  practice. 

Tl^e  present  method  of  using  the  product  form  for  the 
representation  of  the  Inverse  of  the  matrix,  also  makes  use  of 
this  simplification.  Again,  provision  could  be  made  for  covering 
the  rare  non-convergent  case,  but  again.  It  does  not  appear  to 
be  worth-while. 

We  shall  now  describe  a  process  by  which  a  square  non- 
singular  matrix  may  be  expressed  as  a  product  of  elementary 
matrices  of  the  form  (2)  below.  This  Is  illustratively  seen 
for  the  simplex  process  which  Involves  a  step-wise  change  of 
basis  matrix,  that  Is  to  say#  two  successive  matrices  differ 
by  only  one  column.  Using  a  notation  consistent  w^th 
let  •  (P  ,  P.  ,  ...  ,  P,  )  denote  the  basis. 

®  J  i 

If,  In  the  next  basis,  Is  to  replace  P.  ,  then  It  Is  easy 

®  J  jf* 

to  show  that 


(1)  ^ 

Where  and  are  elementary  matrices  related  by 

where  Is  a  ujilt  vector  with  unity  In  the  (i-t-l)  component, 

Is  a  vector  whose  components  are  related  to  components  y^^^ 
ol  by 

^  +  *■ 


•  •  • 


n 


vl 


i/y 


r/ 
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where  It  le  neceeeary  that  ^  0  and  la  defined  by 

(4) 

Successive  applications  of  (l)  for  k-2^  ...  ,  1 


yield 

(5)  ... 

where  B^^^  Is  the  Initial  basis.  It  Is  usually  easy  to  arrange 
that  the  Initial  basis  B^®^  be  the  Identity  matrix  so  that  B^^^ 


■ay  be  dropped  from  (3). 

Consider  the  problem  of  computing  a  row  vector  defined 


by 

(6)  -.^E^  E^j  ...  E,, 

Where  d(  la  a  given  row  vector  (actually  a  unit  vector  in  [l]»[3]  )• 
Such  a  vector  la  required  by  the  revleed  simplex 
■ethod  aa  the  flrat  step  In  detenilnlng  the  vector  P^  to  introduce 
Into  the  basis.  It  Is  clear  that  can  be  obtained  by  successive 
transformations  on  row  vectors,  l.e.,  forming  (c^E^t 
etc.  However,  when  a  row  vector  A  ■  (a^,ai,...,ajg)  Is  trans¬ 
formed  Into  a  row  vector  B  »  (bQ,b| . . . ,b^)  by  multiplying  A  on 
the  right  by  an  elementary  matrix  one  obtains  simply 

'»i  ■  »i  i  j 


(7) 


where^  be 


m 


i 


U  because 
Consiaer  nex 


o 

r  may  be  different 
3xt  the  problem  of 


for  differ, 
computing 


(3)  Y  .  BTlp^  -  . 

It  Ij  clear  that  Y  can  be  determined  by  successive  transfonnatlons 
on  column  vectors,  l.e.,  forming  Ei(P3 etc.  However,  when 
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a  column  vector  C  ■  j^Co>  Ci,...,CQj  is  transformed  into  a 
column  vector  D  «  ^do,dy  • • • by  multiplying  C  on  the  left  by 
a  matrix  of  the  special  form  £  ^  one  obtains  simply 

di  -  cj  *nu\  *  ♦ 

(9) 

From  (7)  and  (9)  it  is  clear  that  the  only  essential  information 
contained  in  is  the  set  of  values  and  the  index  Note 
further  that  in  (6),  the  successive  ^  are  used  with  increasing  ^ 
and  it  follows  from  (9)  that  it  is  necessary  to  know  ^  before 
using  theyf^£.  On  the  other  hand,  in  (6),  the  are  used  in 
decreasing  sequence  of^  but  from  (7)  it  is  not  necessary  to 
know  until  after  the  have  been  used.  The  perfect  comple¬ 
mentarity  of  the  preceding  two  sentences,  together  with  the  fact 

that  can  obviously  be  computed  starting  with  i  ■  m  as  well 

as  with  i  *  0,  makes  it  clear  that  (6)  may  be  computed  using  the 
information  in  the  reverse  order  of  that  used  in  (8). 

Let  denote  the  ordered  set  of  'words'  of  information 

(10)  ••• 

Then  each  change  of  a  column  of  B  will  produce  a  new  which 
may  be  stored  in  consecutive  order  to  the  previously  computed 
Lf  ,L2  ,  •  •  •  ,L^  • 

On  the  CPC,  by  punching  two  sets  of  instructions  on  each 
card — one  being,  in  form,  the  reflection,  in  the  vertical  center 
line,  of  the  other  (with  appropriate  adjustments  for  difference 
in  algorithms  (7)  and  (9|)-~the  transpose  use  of  the  inverse  may 


be  accomplished  by  simply  turning  the  cards  over  using  the  vertical 
center  line  of  the  card  for  the  axis. 


1 
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